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Abstract: The Pierre Auger Observatory is a facility built to detect air showers produced by 
cosmic rays above 10 17 eV. During clear nights with a low illuminated moon fraction, the UV 
fluorescence light produced by air showers is recorded by optical telescopes at the Observatory. 
To correct the observations for variations in atmospheric conditions, atmospheric monitoring is 
performed at regular intervals ranging from several minutes (for cloud identification) to several 
hours (for aerosol conditions) to several days (for vertical profiles of temperature, pressure, and 
humidity). In 2009, the monitoring program was upgraded to allow for additional targeted mea- 
surements of atmospheric conditions shortly after the detection of air showers of special interest, 
e. g., showers produced by very high-energy cosmic rays or showers with atypical longitudinal pro- 
files. The former events are of particular importance for the determination of the energy scale of 
the Observatory, and the latter are characteristic of unusual air shower physics or exotic primary 
particle types. The purpose of targeted (or "rapid") monitoring is to improve the resolution of the 
atmospheric measurements for such events. In this paper, we report on the implementation of the 
rapid monitoring program and its current status. The rapid monitoring data have been analyzed 
and applied to the reconstruction of air showers of high interest, and indicate that the air fluores- 
cence measurements affected by clouds and aerosols are effectively corrected using measurements 
from the regular atmospheric monitoring program. We find that the rapid monitoring program has 
potential for supporting dedicated physics analyses beyond the standard event reconstruction. 
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1. Introduction 

The Pierre Auger Observatory, located about 1 400 meters above sea level near the town of Malar- 
giie, Argentina, is designed to observe extensive air showers created by cosmic rays with energies 
above 10 17 eV. Multiple complementary air shower detectors are operated at the Observatory to 
overcome the shortcomings of any single measurement technique. 

The primary instrument of the Pierre Auger Observatory is a large-area Surface Detector 
(SD) []I], 01, which is used to sample the secondary particles from air showers that reach the ground. 
The SD is an array of about 1 600 water Cherenkov stations arranged 1.5 km apart on a triangular 
grid. The array is deployed over an area of 3 000 km 2 , and it has a duty cycle of nearly 100%. 
Thus, data from the SD provide a high-statistics sample of air showers used to study the energy 
spectrum and arrival direction distribution of the cosmic rays above 10 17 eV. 

While the SD is sensitive to the lateral distribution of secondary air shower particles at ground 
level, the longitudinal development of showers in the atmosphere is measured using a Fluores- 
cence Detector (FD) of 27 optical telescopes [TJ]. The telescopes, optimized for the near- ultraviolet 
band, are located at four sites on the periphery of the SD array: Los Leones, Los Morados, Loma 
Amarilla, and Coihueco (see Fig. [I]). Each site is instrumented with six telescopes deployed inside 
a climate-controlled building. Together the six telescopes have a field of view covering 180° in 
azimuth and about 0° to 30° in elevation. At Coihueco, three additional High-Elevation Auger 
Telescopes (HEAT) have been deployed to observe elevation angles between 30° and 60° @]. 

The fluorescence telescopes are capable of recording the ultraviolet fluorescence and Cheren- 
kov light produced during air shower development. The flux of fluorescence photons from a given 
point in an air shower track is proportional to dE/dX, the energy loss per unit slant depth X of 
traversed atmosphere flSfl. The Cherenkov emission is proportional to the number of charged par- 
ticles in the shower above the Cherenkov production threshold, and depends on the energy loss 
and energy distribution of secondary electrons and positrons in the shower. By observing the UV 
emission from an air shower, it is possible to observe the energy loss as a function of X and make 
a calorimetric estimate of the energy of the primary particle, after correcting for "missing energy" 
not contained in the electromagnetic component of the shower [01 . The slant depth at which the 
energy deposition rate dE/dX reaches its maximum value is called X max . By observing Z max for 
a large set of air showers, the FD data can be used to discuss the composition and the interaction 
properties of cosmic rays as a function of primary energy [0]. 

Simultaneous measurements of air showers with the FD and SD are called hybrid events. 
By performing a joint reconstruction which uses geometrical and timing information from both 
detectors, it is possible to significantly improve the angular and energy resolution of reconstructed 
hybrid events with respect to showers observed by the FD alone 031. Therefore, when FD data 
are used to produce physics results, only hybrid events are included in the analysis. Moreover, 
events observed with high quality in hybrid mode are crucial for the calibration of measurements 
performed using the SD. While the energy of a primary cosmic ray can be estimated using data 
from the SD alone, the absolute scale of the energy estimator depends on hadronic interaction 
models of air shower development. To remove this model dependence, the energy scale of the SD 
is calibrated using a subsample of the hybrid events in which a calorimetric energy measurement 
from the FD can be compared to an independent energy estimate from the SD [0]. 
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The FD is only operated during nights when UV light from air showers is not overwhelmed by 
moonlight. Safe telescope operations also require adequate weather conditions (i.e., no rain and 
moderate wind) and high atmospheric transmittance to assure high data quality. These restrictions 
limit the duty cycle of the FD to about 12% IffOH . As a result, at trigger level the number of events 
observed with the FD is an order of magnitude smaller than that observed with the SD. 

The light profiles recorded with the fluorescence telescopes must be corrected for UV attenu- 
ation along light paths of up to 40 km. To estimate the attenuation of light by molecules, aerosols, 
and clouds, regular atmospheric measurements are performed at the Observatory using UV laser 
shots, radiosonde launches, optical observations, and cloud measurements in the mid-infrared r lTTll . 
The radiosondes provide measurements of the main atmospheric state variables such as tempera- 
ture, pressure, and humidity, which affect mainly the production of fluorescence light induced by 
air showers [0, Pi . but also the light scattering by molecules. The laser shots and optical observa- 
tions are used to estimate the aerosol optical depth and the cloud cover over the FD buildings. 

The regular atmospheric monitoring performed at the Observatory provides atmospheric data 
of local conditions with a time resolution of several minutes to several days, depending on the 
type of measurement. This is sufficient for the bulk of measured air showers. Hourly and daily 
atmospheric corrections are available for reconstructing individual showers, and the average en- 
ergy dependence of the atmospheric corrections for the full sample of observed cosmic rays is 
well-understood [OH . However, because of the massive volume of atmosphere used to perform 
fluorescence observations - nearly 30 000 km 3 - the time and spatial resolution of the atmospheric 
database is necessarily limited. 

For some analyses, it is desirable to provide atmospheric data beyond the regular measure- 
ments. For example, the high-energy tail of the data sample used in the SD energy calibration is an 
important lever arm in the SD-FD fit. Since atmospheric corrections are of utmost importance for 
the highest-energy showers recorded with the FD, it is sensible to perform dedicated atmospheric 
measurements at the time and location of high-energy cosmic ray events. Other showers of interest 
are anomalous longitudinal profiles observed in the FD data. The rate of these showers is expected 
to be largest at low energies and for light primary masses fOl . Such showers are removed by stan- 
dard analysis cuts because lumpy profiles are typically caused by atmospheric non-uniformities 
such as cloud banks or aerosol layers. However, these profiles may also be indicators of exotic 
primary particles or unusual air shower development. In any analysis which uses longitudinal pro- 
files to search for such exotic phenomena, dedicated monitoring of air-shower tracks is needed to 
remove events which could be distorted by atmospheric effects. 

To provide high-resolution atmospheric data for interesting air showers, we have implemented 
an automatic online monitoring system which can be used to trigger dedicated atmospheric mea- 
surements a few minutes after the air showers are detected. This rapid monitoring trigger was 
commissioned in early 2009 and has been integrated into the regular monitoring schedules of sev- 
eral of the atmospheric monitoring subsystems. In this paper, we will discuss the operation and 
performance of the rapid monitoring program. In Section @, we describe the Pierre Auger Ob- 
servatory and review the standard atmospheric monitoring program. The implementation of the 
online atmospheric monitor is discussed in Section [3[ The integration of rapid monitoring into the 
radiosonde, lidar, and optical telescope subsystems is discussed in Sections |], |5], and ^, along with 
a selection of interesting showers. We conclude in Section [7[ 
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2. Atmospheric Monitoring 



As described in Section [I], measurements of air showers with the fluorescence telescopes are af- 
fected by fluctuations in atmospheric conditions, and so extensive atmospheric monitoring is car- 
ried out at the Observatory [0111. The locations of the SD, FD, and the atmospheric monitors 
described in this work are shown in Fig. PJ. 



Figure 1. Layout of the surface detector array (dotted area) and fluorescence telescope sites, showing 
only the locations of the atmospheric monitoring subsystems which are integrated into the rapid monitoring 
program. The two central laser facilities, which do not receive rapid monitoring triggers, are shown for 
reference. 

Atmospheric measurements are stored in several multi-gigabyte databases for use in the offline 
reconstruction of air showers. The time resolution of the measurements ranges between five min- 
utes (in the case of cloud data) to one hour (in the case of aerosol data) to several days (in the case 
of altitude-dependent atmospheric state variables). The spatial resolution is limited, the altitude- 
dependent atmospheric state variables are assumed to be horizontally uniform across the SD array, 
while aerosol conditions and state variables from ground-based weather stations are treated as uni- 
form in the region around each FD building or station, respectively. The systematic uncertainties 
introduced by the limited resolution of the database have been estimated and are reported as part of 
the uncertainty in the FD energy scale provided for the SD energy calibration f P . IPtl . Due to the 
correlation between the reconstructed energies of air showers and the distances at which they are 
observed in the telescopes, the uncertainties increase linearly with energy WHW . 

2.1 Atmospheric State Variables and Site Models 

Air temperature, pressure, wind speed, and humidity are recorded at ground level by weather sta- 
tions at each FD building and at the Central Laser Facility (see Fig. PJ), and between 2002 and 2010 
a weather balloon program was operated at the Pierre Auger Observatory. Prior to mid-2005, the 
radio soundings were performed in ten dedicated campaigns, each lasting two to three weeks, with 




FD Loma Amarilla 
Lidar 



FD Los Morados 



FD Los Leones 
Lidar, FRAM 



8 




altitude (km a.s.l.) altitude (km a.s.l.) altitude (km a.s.l.) 




altitude (km a.s.l.) altitude (km a.s.l.) altitude (km a.s.l.) 



Figure 2. Top: Atmospheric profiles of the new Malargiie Monthly Model, left: temperature, middle: 
atmospheric depth, right: water vapor pressure. The depth profiles are expressed with respect to the U.S. 
Standard Atmosphere / TO? . Bottom: The same graphs as in upper row with uncertainties for February (solid 
line with yellow uncertainties) and August (dashed line with blue uncertainties). 



an average of 10 launches per campaign. Between mid-2005 and end of 2008, the balloon launches 
were performed more regularly - about every five days and independently of FD data-taking. 

To compensate for the missing information between the radiosonde measurements, average 
models of monthly conditions were constructed. The first version of these Malargiie Monthly 
Models (MMM) contained vertical profiles of atmospheric temperature T, pressure p, density p, 
and atmospheric depth X derived from pre-2005 weather data from Malargiie and data from Cor- 
doba and Santa Rosa, Argentina, the sites nearest Malargiie with publicly available radio sounding 
measurements [ITBl . The local measurements were supplemented with external data because of the 
low measurement statistics at the Observatory when the models were constructed. By 2009, the 
number of balloon flights over the Observatory was sufficient to re-evaluate the profiles and con- 
struct improved models with an additional average profile of the water vapor pressure e WTfW . These 
new Malargiie Monthly Models (nMMM) were derived from 261 local radio soundings performed 
between August 2002 and December 2008. 

The nMMM profiles comprise vertical profiles of T, p, p, X, and e specified between 1.2 km 
and 30 km above sea level in steps of 200 m. Of the 261 radio soundings used to construct the 
models, 32 were discarded during construction of the vapor pressure profiles due to contamination 
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of the balloon flights by high cloud coverage. Above 12 km, the vapor pressure has been set to 
zero. 

The local radio soundings provide reliable and unbiased measurements of the monthly average 
profiles between about 1.6 km and the burst altitude of the balloons. The burst altitude was typically 
at 23 km, with a few balloons reaching a maximum altitude of 27 km. Data from the five ground- 
based weather stations at the Observatory were used to extrapolate the profiles down to 1.2 km 1 . 
Above the altitude of balloon burst, the data have been extrapolated using values from the 2005 
monthly models. The nMMM profiles of T, X, and e are shown in Fig. @, top row. 

The uncertainties of the model atmospheres are quite large. For temperature, the RMS fluc- 
tuations at ground level range between 3 K during austral summer to 6 K during austral winter 2 ; 
at 26 km, the RMS spread is 0.5 K during austral autumn and 5.0 K during austral spring. Atmo- 
spheric depth varies mainly between 4 km and 8 km. The RMS spread of atmospheric depth at 
ground ranges between 2 g cm~ 2 (summer) and 5 g cm -2 (winter); the largest RMS, at 8 km, is 
about 7.5 g cm~ 2 . Above 18 km, the depth uncertainties are below 1.5 g cm~ 2 . The vapor pressure 
RMS at ground is 1.5 hPa (summer) and 4.0 hPa (winter), but is well below 0.2 hPa above 7 km. 
For illustration, the uncertainties are plotted exemplarily for February (austral summer) and August 
(austral winter) in Fig. @, bottom row. 

2.2 Optical Transmission and Cloud Detection 

During the 15 to 19 nights per lunar cycle that are dark enough to operate the fluorescence tele- 
scopes, hourly measurements of the aerosol optical depth fllTl . H7H are made as a function of altitude 
with two central laser facilities HT8H and four lidar stations HT911 . In addition, an optical telescope 
called the ph(F)otometric Robotic Atmospheric Monitor (FRAM) [EH is used to measure the in- 
tegral aerosol optical depth inside and outside the field of view of the FD building at Los Leones. 
Finally, the cloud coverage at the Observatory is measured with the lidar stations and infrared 
cameras located at each of the four FD sites [PH . 

There are four lidar stations, one per FD site, and during regular operations the lidars are 
used to scan the atmosphere outside the field of view of the FD telescopes. Currently, the scans 
are used to retrieve the mean cloud cover and the lowest cloud height during each hour of FD 
measurements. IR cloud cameras provide complementary 2D images of the whole field of view 
every five minutes fTTHl . A direct combination of these two pieces of information is used to provide a 
three-dimensional map of clouds above the Observatory, but not without ambiguities. For instance, 
inspection of the lidar data has shown that multiple cloud layers are present above the site about 
30% of the time; a mismatched altitude may be associated to the clouds detected by the IR cameras 
since different cloud layers cannot be easily distinguished in the IR images. 

FRAM is a robotic optical telescope with primary mirror diameter of 0.3 m located about 30 m 
from the fluorescence detector building at Los Leones. The instrument was installed primarily to 
determine the wavelength dependence of the extinction caused by Rayleigh and Mie scattering. 
This goal is achieved using the photometric observations of selected standard (i .e. non-variable) 
stars, and recently also using the photometric analysis of CCD images. The results of this primary 

1 For technical reasons during air shower reconstruction, the profiles need to go beyond the lowest surface height. 

2 Austral summer refers to the months of December, January and February, austral winter corresponds to June, July 
and August. 
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mission are presented in flZTH . Since its installation in 2005, the FRAM telescope has also been 
involved in automatic observations of optical transients of gamma-ray bursts. This program is very 
successful and several light curves of transients were already observed, including one uniquely 
bright GRB afterglow |JZ2|. 

3. The Rapid Atmospheric Monitoring Program 

Atmospheric uncertainties grow as a function of primary particle energy because of the energy de- 
pendence of the longitudinal development of air showers, which affects the geometry of observable 
showers within the field of view of the FD fllTTl . An improvement in the resolution of the atmo- 
spheric monitoring data can be achieved by triggering measurements of the atmosphere within a 
suitable time interval after the detection of high-energy showers above a certain threshold (e. g., 
E > 10 19 eV). Such triggers have been implemented for the individual weather balloon, lidar, and 
FRAM optical telescope subsystems. 

During the FD data taking, an automated process is used to collect event data from the FD 
and SD, build and reconstruct hybrid events, and send the reconstructed shower parameters to the 
atmospheric monitoring subsystems participating in the rapid monitoring program. Each subsys- 
tem performs individualized cuts on the shower parameters, and if the shower is a candidate for 
special monitoring - e. g., it has a well-observed track and is of a particularly high energy - an 
atmospheric measurement is performed either in the vicinity of the shower track (for transmission 
measurements) or above the Observatory with meteorological weather balloons. In this manner, 
the time resolution of the atmospheric measurements can be reduced from hours to minutes (for 
the lidars and the FRAM) or from days to hours (for the weather balloons) with respect to the ar- 
rival time of an interesting shower. Moreover, the lidar stations and the FRAM are able to directly 
probe the atmosphere along the shower-detector plane - the plane defined by the position of the 
FD telescope and axis of the shower - reducing the uncertainties introduced by the assumption of 
horizontally uniform atmospheric layers in the weather databases. 

The rapid atmospheric monitoring system consists of three components: an online event 
builder that merges shower data as they are sent to the Observatory campus in Malargiie; a hy- 
brid reconstruction that uses all the detector and calibration data that are available immediately 
after a shower is detected; and a broadcast program that notifies the atmospheric subsystems of the 
detection of a hybrid event. The programs are designed to run without human intervention during 
FD measurements. We discuss the software components in Section ^71] and review the performance 
of the reconstruction in Section E3- 

3.1 Online Event Builder, Reconstruction, and Broadcast 

The flow of data between the Observatory campus and the atmospheric subsystems is shown in 
Fig. 0. During FD measurement periods, data from the fluorescence telescopes are transferred to 
Malargiie in a 20-second cycle. Simultaneously, triggers and recorded data from the surface array 
are sent to the SD Central Data Acquisition System (CDAS), a computer cluster and disk array 
located in Malargiie. Due to a polling delay that allows the SD communications system to collect 
data from across the array, surface station data typically arrive in the CDAS 2 to 8 minutes after the 
detection of an air shower. 
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Figure 3. Network diagram of the rapid atmospheric monitoring system. Data from the fluorescence and 
surface detectors are transferred to a Reconstruction PC in Malargiie. The data are merged, reconstructed, 
and sent to the atmospheric monitoring PCs (Lidar PC, Balloon PC, and FRAM), where triggers are formed 
and sent to the monitoring devices. The Lidar PC also inhibits FD data acquisition during laser shots by 
sending a veto request to the FD GPS clock modules (see Section \5. Lip . 



Once the SD and FD data are available in Malargiie, a fast online event builder produces air 
shower data in the standard hybrid format. The data are reconstructed using a version of the Auger 
reconstruction software AZ3I1 . which is named Off line modified for online running. The online 
reconstruction is configured to use the latest available detector and calibration databases, and it 
is kept in sync with releases of the Off line software. This is to keep the results of the online 
reconstruction as close as possible to the standard offline 3 reconstruction. However, since the non- 
event databases are typically updated on timescales of 4-6 months, some drifts between the online 
and offline reconstructions are unavoidable. 

In the offline reconstruction, large -particle scattering by aerosols is estimated using atmo- 
spheric measurements. It is not possible to use real-time atmospheric monitoring data in the online 
reconstruction, so instead an average parametric model of aerosol scattering in Malargiie is used. 
Rayleigh scattering by molecules is calculated using the nMMM average monthly models. The 
systematic uncertainties introduced by the use of average models is discussed in Section 

Approximately 80 geometry, quality, and energy parameters from each reconstructed shower 
are written to disk on the Reconstruction PC (cf. Fig. 0). As they are saved to disk, the events 
are also transfened to the atmospheric monitoring subsystems (Balloon PC, Lidar PC, and FRAM) 
via network broadcast. Client programs in the atmospheric monitors are used to perform cuts on 
the hybrid data and issue triggers based on the specialized measurements performed with each 

3 While Off line wl \\ refer only to the software framework, "offline" is meant to describe processes that happen several 
days to months after the measurement. 
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Figure 4. The timing scheme of the rapid monitoring system. Top: the timeline of rapid monitoring with the 
lidar stations (L), FRAM (F), and the balloon launches in the context of standard operations. Atmospheric 
scanning with the lidars and FRAM is typically completed within 20 minutes of the detection of an event. 
The balloon launches are initiated about 90 minutes after the event. Bottom: a detailed view of the first 20 
minutes after the detection of an event. Events are reconstructed within 2 to 9 minutes of the initial detec- 
tion and the results are broadcast to the monitoring PCs. Each subsystem applies an individualized trigger 
criterion (T) to identify showers for follow-up monitoring. In the case of the lidar and FRAM telescopes, the 
regular operations are interrupted and the telescopes slew into position to begin a scan of the shower-detector 
plane. During the lidar scan the FD DAQ is inhibited by a veto to avoid spurious triggers caused by scattered 
laser light. Regular atmospheric sweeps resume once the scans are complete. In the case of the balloon 
system, a text message is sent to an on-site technician who drives to the balloon launch facility, prepares the 
balloon, and starts the radiosonde measurement. For more details, see SectionsQQ, and@. 



instrument (see Sections g], |5], and 0). 

In Fig. |], a timing diagram is shown for the online reconstruction and the activity within all 
three subsystems. More details on the individual steps are provided in the corresponding sections. 
It should be noted that the online reconstruction runs continuously. The pictured timeline shows 
only the case if an interesting air shower event is identified by subsequent steps. Also, the three 
systems operate independently, they do not necessarily trigger on the same air shower event because 
of different trigger criteria. 

3.2 Reconstruction Performance 

We illustrate the performance of the online reconstruction using hybrid data recorded between 
March 2009 and March 2011. During this period, 320 hybrid events reconstructed online had 
energies above 10 19 eV and passed standard quality cuts based on the event geometry and dE/dX 
profile fit [0, Applying the same cuts to data reconstructed offline produces a set of 382 events 
during the same period. 
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Figure 5. Comparison of shower energy and position of shower maximum reconstructed online and offline. 



Note that AE = E ofnjne - E online and AX max = X. 



offline 



X 



online 



(E) is the average of the two reconstructed 



energies. Real hourly aerosol measurements were used in the offline 



Inspection of the events shows that the online and offline sets have only 255 events in common. 
The discrepancy, and the lower number of online events, is caused by several factors. The number 
of events reconstructed online is reduced by downtime in the online reconstruction due to various 
technical problems such as software failures, network crashes, etc. For example, the downtime of 
the online reconstruction during 2010 was about 15%, which accounts for much of the difference 
in size between the online and offline event samples. In addition, most of the offline data were 
corrected for real aerosol conditions, whereas the online reconstruction uses an average model 
of aerosols above the Observatory. The shower profiles reconstructed online tend to be of worse 
quality because true aerosol scattering is not taken into account, and so more events fail the offline 
quality cuts on the shower profile. The migration of events around the quality cuts due to changes 
in the software versions and databases used in the reconstruction also accounts for an additional 
reduction in the number of events in common between the online and offline data sets. 

It is instructive to compare the common events of the two data sets. In Fig. |5] the differences 
in the energy and X max of the common events are plotted. Both distributions contain significant 
tails, and the energy reconstructed online is systematically higher than the energy reconstructed 
offline by about 7%. The main cause is the lack of true aerosol corrections in the online data, 
which accounts for at least half the offset between the two reconstructions [HTj] . The remainder of 
the offset is due to differences in software versions between the online and offline reconstructions 
and the lack of nightly calibration constants in the online reconstruction. 

Even though the online reconstruction is affected by a non-negligible downtime, it appears to 
have performed well since it was first implemented in 2009. The comparison between the online 
and offline events indicates the presence of a significant systematic bias in the online data because 
of the use of an average aerosol model. This means that some events which pass the online cuts may 
not survive the offline analysis cuts. In the absence of real-time aerosol data this is unavoidable. 
However, it may be possible to tune certain measurements using nearly real-time conditions and 
hence reduce "false positive" triggers. An example application is discussed in Section |5.2.2| . 
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4. Balloon-the-Shower Program 

The use of monthly site models to estimate atmospheric state variables and molecular scattering 
rather than real-time radiosonde data introduces an uncertainty into the estimated production and 
transmission of fluorescence light in air showers. This contributes to the statistical uncertainties 
in the reconstructed energy and position of shower maximum. The total effect is moderate, but 
it does depend on the shower energy. Between primary energies of 10 17 ' 7 eV and 10 eV, the 
monthly profiles contribute 1.5% (at 10 17 ' 7 eV) and 3% (at 10 20 eV) to the total energy resolution 
of about 8% E3, and 7.2-8.4 g cm~ 2 to the total X max resolution of about 20 g cm~ 2 07J] of the 
hybrid reconstruction HTT1 . IT2I . It is important to note that these numbers are characteristic of a 
large sample of showers, but the systematic errors in the reconstruction of individual showers can 
be substantially larger, particularly at high energies. Therefore, it is desirable to minimize as much 
as possible the atmospheric uncertainties in the reconstruction of high-energy events. 

To improve the resolution of the reconstruction for the highest-energy showers, the Balloon- 
the-Shower program (BtS) was operated between March 2009 and December 2010. Its purpose 
was to perform an atmospheric sounding within about three hours of the detection of a high-quality 
high-energy event. 

4.1 Performance of BtS 

In March 2009, BtS replaced regularly scheduled meteorological radio soundings at the Observa- 
tory. The target launch rate was chosen to be three to seven launches per FD measurement period, 
with each period lasting about 2.5 weeks. The focus of the BtS program was high-energy showers 
used in the SD energy calibration or the hybrid mass composition analysis; in other words, hybrid 
events with well-reconstructed longitudinal profiles and energies above 10 19 eV. 

The atmospheric profiles from the BtS program represent an independent data set that can be 
compared to the nMMM average models. The difference between each BtS profile and its corre- 
sponding nMMM profile is plotted in Fig. |[ The width of the deviations is in agreement with the 
uncertainties of the monthly models described in Section ^J]. 

Events passing the online cuts were used to trigger a text message sent to an on-site technician, 
who then drove to the Balloon Launch Station to launch a weather balloon. Given the lack of 
automation, the radiosonde flights typically took place only several hours after the detection of a 
cosmic ray event. To minimize the delay, it was decided to limit the time difference between event 
detection and balloon launch to a maximum of three hours. This delay was not expected to affect 
the validity of the radiosonde data, since fluctuations in the vertical atmospheric profiles tend to be 
much larger between nights than within a single night []2"7ll . 

4.1.1 Quality Cuts 

To trigger a BtS launch, showers from the online reconstruction were required to pass quality cuts 
used in publications of the SD energy spectrum \UM and the hybrid mass composition [0]. The cuts 
are listed in Table [I] and were designed to minimize the uncertainty in shower energy and X max . 
In fact, the cuts used for BtS are moderately stricter than those used in 071 H to account for the 
systematic uncertainties in the online reconstruction described in Section fJ72| . 
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Figure 6. Atmospheric profiles obtained from launches within the BtS program for temperature (left), 
atmospheric depth (middle), and water vapor pressure (right). Each actual profile is shown in difference to 
the according profile from the nMMM. There are no launches for January and February. 



Cuts (1) and (2) select showers in which the reconstructed energy and the position of shower 
maximum are reliably estimated. Cut (3) removes showers in which X max is less than 20 g cm~ 2 
from either the minimum or maximum observed depth of the shower track. This reduces the pos- 
sibility that X max is mis-identified and also improves the reconstructed dE/dX profile. Cut (4) is a 
standard geometry cut that ensures the surface station with the largest signal (i. e., the one used for 
timing in the hybrid reconstruction) is close to the shower core. Cuts (5) and (6) are indicators of 
the quality of a Gaisser-Hillas parametric fit to the longitudinal shower profile [0, EBD. Cut (5) is 
effective at removing showers obscured by clouds and other atmospheric non-uniformities, since 
the profiles of such showers deviate strongly from a Gaisser-Hillas curve. This cut also removes 
possible exotic air shower candidates that are covered by the other two subsystems (cf. Sec. |5.1.3| 
and Sec. |6.3.2 ), as they are not the main focus of the BtS program. Cut (6), a x 2 difference between 
a linear fit and a Gaisser-Hillas fit to the longitudinal profile, removes faint, low-energy showers 
from the trigger sample. The fraction of rejected events of the cuts are included in Table |TJ. 

The hybrid events recorded between January 2006 and January 2009 were used as a tuning 
sample to set the rate of BtS triggers. The effect of the cuts are shown as a function of energy in 
Fig. 0. To reach the desired number of atmospheric soundings - 50 to 60 per year, or about 3 to 7 
per FD measurement period - it was necessary to further reduce the size of the event sample with a 
cut on the reconstructed shower energy. The energy threshold, determined from the tuning sample, 
was 



logm ( ) = 19.3 O £ min = 19.95 EeV. (4.1) 
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Rejected Events 


(1) Energy uncertainty 


o E /E < 0.2 


47.5% 


(2) X max uncertainty 


Oy < 40 e cm z 


59.5% 


(3) Field of view 


X max well observed 


8 Set, 

O.O70 


(4) Distance to SD with highest signal 


daxis < 1500 m 


0.5% 


(5) Quality of Gaisser-Hillas (GH) fit 


XGn/ n doi < 2.5 


5.3% 


(6) Comp. of GH with linear fit 


Xlin ~ %GH > ^ 


36.8% 


Energy threshold 


E Q > 19.95 EeV 


99.3% 



Table 1. A list of quality cuts for the BtS program. The fraction of rejected events were calculated using 
hybrid data recorded from 2006 to 2009. Note that the percentage in each row is given with respect to the 
previous cut. In the last line, all events below the energy threshold were discarded. 
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Figure 7. Candidate event distribution for BtS estimated by using hybrid events collected between 2006 
and 2009. The effect of the quality cuts and the energy threshold is shown. Note that the two profile fit cuts 
(5) and (6) were combined. 

4.1.2 Trigger of Weather Balloon Launches 

After a shower passed the automatic BtS trigger, a short message containing the date and time of 
the event was sent to the mobile phone of an on-site technician. If the message was received, the 
technician would drive to the Balloon Launch Station and proceed with the atmospheric sounding 
within three hours of the event. 

Between March 2009 and December 2010, 100 text messages were sent to the technician. 
From these 100 triggers, 52 balloons were launched successfully. Some messages were received 
while a radiosonde was already in flight, due to the tendency of high-quality, high-energy obser- 
vations to cluster during very clear, cloudless nights. Therefore, 62 BtS triggers were covered by 
the 52 flights. The remaining triggers, about one-third of the total, were lost due to technical issues 
such as a hardware failure at the Balloon Launch Station in August 2009 (11 events), problems 
with the transmission of the text messages, or other failures in the radiosonde flights. 

The BtS statistics between March 2009 and December 2010 are shown in Fig. Note that 
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Figure 8. Expected BtS triggers for the 22 measurement periods between March 2009 and December 201 
using the offline reconstruction / El7 and BtS cuts (Table |7|). Also shown in grey is the number of generated 
text messages for those expected events. Shown in red is the number of events that were covered by a balloon 
launch. No visible white bar means every interesting event was caught by the system. No visible grey bar 
means there was a launch for every text message. 



the chart shows events reconstructed using the offline reconstruction (and after quality cuts were 
applied) and not the online reconstruction. During the period of BtS operations, 88 events re- 
constructed offline passed the BtS cuts. Of these, 59 were also identified online, meaning that 
out of the 100 events which triggered a text message, 41 do not survive the BtS cuts after offline 
reconstruction. Some did not satisfy the BtS cuts, while others fell below 10 19 eV. Of the 59 
common events, 35 were covered by a weather balloon launch. If the energy cut is relaxed slightly 
to 10 19 - 2 eV, 51 offline events are covered by a balloon launch. 

4.2 Air Shower Reconstruction using BtS Data 

To evaluate the effectiveness of the BtS program, we have reconstructed hybrid events covered by 
the radiosonde launches conducted since March 2009. The results are compared to a reconstruction 
that uses the nMMM average monthly conditions, as well as more real-time conditions given by the 
Global Data Assimilation System (GDAS) HZ9H . GDAS is a global atmospheric model based on 
meteorological data and numerical weather predictions. Altitude-dependent profiles of atmospheric 
state variables such as T, p, and e are provided on a 1° x 1° latitude-longitude grid. The GDAS 
database contains profiles which are useful for the needs of the Pierre Auger Observatory with 
a time resolution of three hours starting in June 2005. Hence, the database covers not only the 
period of BtS launches, but also most of the period of data-taking at the Auger Observatory. GDAS 
data were made available to the air shower analysis of the Auger Observatory beginning in spring 
2011 \M- 

4.2.1 Effect of BtS Profiles and Model Atmospheres on the Reconstruction 

To study the effect of the BtS data on the reconstruction of air shower profiles, we have recon- 
structed the 62 hybrid events covered by 52 BtS launches. This data sample contains 52 events 
which pass all quality cuts consisting of 90 individual fluorescence profiles after accounting for 
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events observed in stereo with multiple telescopes. We also compared the reconstruction using BtS 
data to those using nMMM and GDAS model profiles. In case of reconstructions using nMMM, two 
more events where discarded by cut criteria, thus only 50 events are contained in this reconstruction 
data sample. In all cases, we have accounted not only for the effects of the atmospheric profiles on 
light scattering, but also for the effects of temperature, pressure, and humidity on fluorescence light 
production 03L O, ED- For the fluorescence light calculation, experimental data from the AIRFLY 
experiment [H2] and conference contributions ffJ3ll from the AIRFLY collaboration were used. 
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Figure 9. Energy distribution of all reconstructed air showers which passed the BtS quality cuts during 
the online analysis and for which a radio sounding has been performed. The black, solid line represents 
the energy distribution of 52 events reconstructed offline with actual atmospheric conditions as measured 
during the dedicated weather balloon ascents, Ssts- The red, dotted line, S„mmm, displays 50 successfully 
reconstructed events and the blue, dashed line - Sgdas - indicates the distribution of the same 52 events 
as in Ssts- The two missing events not covered in the offline reconstruction S„mmm are due to onset of a 
cut criterion or due to failed reconstructions because of extreme shower geometries. The distributions agree 
very well within the expected uncertainties. 



In the study, we devote our main attention to the reconstructed cosmic ray energy E and depth 
of shower maximum X max of the air shower profiles. The energy distribution of the reconstructed 
events is provided in Fig. 0. For air showers detected by more than one telescope, the weighted 
mean of the shower observables is used. All events represented by the solid line are reconstructed 
using the atmospheric profiles gathered within the BtS program, Ssts- Reconstructions applying the 
monthly atmospheric conditions as described by nMMM, S n MMM> are displayed with a dotted red 
line. The third set of reconstructed air showers, Sgdas, is plotted as a dashed blue line and was ob- 
tained using the corresponding model atmospheres from GDAS. The three distributions agree well 
within the systematic uncertainty of the hybrid energy reconstruction l1h4Tl . The overall systematic 
uncertainty is 22%, whereof 1% are contributions due to atmospheric uncertainties [@] which are 
discussed here. Note that some events have spilled below the energy threshold of 10 19 ' 3 eV because 
of the systematic energy shift between the online and offline reconstructions. The mean energy of 
the event sample is 10 194 eV. 

In Fig. [R)| and Fig. ITT), the distributions of the energy difference AE / (E) and AX max between 
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Figure 10. Energy difference between events se- 
lected by the BtS program. The air showers are 
reconstructed with three different atmospheric de- 
scriptions, for details see text. 



Figure 11. Difference of the position of shower 
maximum X max between events selected by the BtS 
program. The air showers are reconstructed with 
three different atmospheric descriptions, for details 
see text. 
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Table 2. The mean differences and RMS values for the comparisons of Sets an d SnMMM as we U as Sets an d 
Sgdas m relative energy (AE / (E)) and position of shower maximum (AX max ). 



the Sets and S^mm events (solid line) and the Sbis and Sgdas events (dotted line) are plotted, 
respectively. The quantity (E) is the average of the energies reconstructed with each pair of atmo- 
spheric profiles. The mean differences and widths of the distributions for both AE/(E) and AX max 
are listed in Table 0. For the BtS-nMMM comparison, the most extreme differences of about 6% 
and 16 g cm are found for AE/(E) and AX max , respectively. The width of the distribution for 
the BtS-GDAS comparison is smaller because the time resolution of the GDAS profiles is much 
finer than that of the monthly models. The comparison indicates that the GDAS data provide a 
reasonable description of the local conditions on time scales of a few hours. 

Finally, it can be concluded that the systematic uncertainties of the energy (22%) and of X max 
(13 g cm -2 07D) are hardly reduced by applying actual atmospheric profiles in the reconstruction of 
extensive air showers instead of applying adequate local models. The resolution of E and X max can 
be slightly reduced by 0.3% and 1.1 g cm -2 , respectively. 

4.2.2 Study of Systematics 

In this section, we describe several possible systematic effects in the event reconstruction: 
1. Energy dependence of the AE/(E) and AX max distributions; 
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Figure 12. Differences in reconstructed energy (left) and X max (right) vs. energy. 
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Figure 13. Reconstruction results vs. month of year. Left: Difference in E. Right: Difference inX n 
There were no balloon launches in January or February. 
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Figure 14. Reconstruction results vs. vertical X max . Left: Difference in E. Right: Difference in X max . 
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2. Seasonal effects; 



3. Dependence on vertical X max . 

We investigate systematic effects using the differences in the reconstructed energy and X max af- 
ter using the BtS, nMMM, and GDAS data in the reconstruction. The energy dependence of the 



distributions of AE / (E) and AX max is shown in Fig. [12|. The Ssts-SnMMM comparisons are plot- 
ted with black points, while the Sbis-Sgdas comparisons are plotted with red squares. No energy 
dependence is observed. 

Seasonal effects have been investigated by plotting the energy and X max differences according 
to the calendar month (Fig. [T3|). While the primary energy does not show any signs of seasonal 
dependence, there are larger fluctuations in X max in particular during the austral winter when using 
the nMMM profiles instead of the BtS profiles in the reconstruction. 

The third systematic dependence of interest is vertical X max (see Fig. [TJ]). Vertical X max is the 
projection of the inclined shower track to the vertical, which we use to correct for the different incli- 
nation angles of the air showers and establish a clear relationship to the layering of the atmosphere. 
For both E and X max , no dependence is obvious. Note that the entry for vertical X max between 750 
and 800 g cm~ 2 corresponds to only one event, since only one air shower profile has been detected 
with such a deep X max after applying all quality cuts of the BtS program. This particular shower 
entered the Earth's atmosphere quite vertically with a reconstructed zenith angle of about 12°. 

Finally, we performed additional searches for any dependence of AE/(E) and AX max on flu- 
orescence detector location, on the distance of the core position from FD telescope, and on some 
further effects induced by the incoming geometry of the air shower. The dependence of AE/(E) 
and AX max on these parameters is negligible in all cases. 



5. Shoot-the- Shower Program 

The purpose of Shoot-the-Shower (StS) is to initiate lidar scans of the shower-detector plane created 
by the image of an air shower in an FD telescope. The motivation is to identify atmospheric non- 
uniformities - especially clouds - that obscure light from the shower as it propagates to the FD 
telescopes. Such non-uniformities may not be present in the hourly atmospheric databases, and so 
StS is intended to supplement the cloud identification performed using the regular lidar scans. 

An hourly cloud coverage below 20% is required for hybrid events to be used in the analysis 
of the mass composition and energy spectrum of the cosmic rays observed at the Pierre Auger 
Observatory [0, EUl- This cut may still allow sparse clouds to affect the FD measurements, so 
one of the main goals of StS is to observe showers which pass the cloud coverage cuts but fail 
the longitudinal profile cuts. StS can be used to verify that the profile quality cuts are removing 
showers contaminated by weather effects and not also removing physically interesting showers 
from the event sample. The StS trigger has also been adjusted to support the search for anomalous 
longitudinal profiles due to hadronic interactions HT3H . These two running modes are described in 
the following sections. 



-22- 



Start 




Figure 15. Simplified state diagram of the lidarrun control. During normal operations, the lidarruns in an 
automatic scan state (AutoScan). When an StS trigger is received, the run control software calculates the 
shooting trajectories for all four lidar stations and attempts to veto the DAQ of the FD telescopes. If the veto 
is set and confirmed, the StS proceeds. After StS terminates, the lidar returns to the AutoScan state. The 
lidar stations will automatically shut down if wind or rain exceed safe operating conditions, or if there is a 
break in network communications between the stations and run control in Malargiie. 

5.1 Performance of StS 

Each lidar station contains a steerable telescope and a 150 uJ Nd:YLF laser with a central wave- 
length of 355 nm. The telescope collects backscattered laser light, and the analysis of the return 
signal can be used to infer the presence of aerosols and clouds along the light path WFX\ . Because 
the laser wavelength is in the center of the UV acceptance window of the FD telescopes [TJ] , the op- 
eration of the lidar must be carefully controlled to avoid triggering the FD telescopes with scattered 
laser light. The implementation of the control system for StS is briefly described in Section |5.1.1| . 
In Sections |5.1.2| and |5.1.3| , we describe the general-purpose and anomalous-profile StS triggers, 
respectively. 

5.1.1 Full-Site Veto for StS 

During normal operations, the lidar stations are programmed to observe the atmosphere above each 
FD building outside the field of view with an automatic scanning mode called AutoScan. When an 
StS trigger is received, the lidar stations must stop AutoScan and sweep through the shower-detector 
plane. This requires the lidars to scan inside the field of view of the FD, creating the possibility of 
spurious "self-triggers", or backscattered laser light triggering a nearby FD telescope, and "cross- 
fires", or forward-scattered laser light triggering a telescope on the other side of the SD array. To 
avoid spurious triggers, we have designed the StS mode to inhibit the FD DAQ for the duration of 
the StS measurement. 

The implementation of the DAQ veto is shown schematically in the state diagram in Fig. |T3j. 
Online hybrid events are broadcast to the lidar run control PC running AutoScan in the Malargiie 
campus (cf. Fig. 0). The run control program analyzes the events for StS trigger conditions. When 
an event passes the triggers, the run control program calculates a scanning pattern for all four 
lidar stations and transmits shooting coordinates to the lidar control PCs at the lidar site. When 
the shooting coordinates are received and automatically confirmed, the run control program sets a 
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(4) Track length 
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4% 


(5) Local zenith angle 


d < 60° 


33% 


(6) FD-core distance 


dFD.core > 5000 m 


1% 



Table 3. A list of quality cuts for the StS program. The fraction of rejected events were calculated using 
hybrid data recorded from January 2006 to August 201 1. Note that the percentage in each row is given with 
respect to the previous cut. The event time cut throws out events which are analyzed by the lidar run conttol 
trigger more than 10 minutes after they were originally detected. 

DAQ VETO bit in the GPS servers at each of the FD buildings. After the veto is set and confirmed 
by the run control software, the AutoScan is paused and the lidar stations begin the StS sweep. 

To prevent the lidar run control from deadlocking the FD DAQ, the GPS server VETO bit 
is set to revert automatically after four minutes, re-enabling FD data acquisition. The lidar StS 
coordinates are calculated such that the lidar telescopes can safely complete the scan during the 
four-minute shooting window. If the lidar run control program in Malargiie loses network com- 
munication with the lidar stations at any time during AutoScan or StS, the stations will revert to a 
partial shutdown mode to prevent uncontrolled laser interference with the FD telescopes. 

5.1.2 General-Purpose StS Trigger 

The first set of quality cuts applied to StS candidate events, given in Table 0, are designed to include 
the showers that are likely to become part of the main high-energy hybrid data set. However, the 
cuts are also loose enough to accept events with unusual "dips" and "spikes" in the longitudinal 
profile. Such features are typically caused by strong attenuation and multiple scattering by clouds 
and aerosol layers. Including these events in the StS sample can allow us to investigate why some 
longitudinal profiles do not pass strict profile cuts. We also considered the possibility that such 
events might be recovered for use in the analysis in the future, if the cloud-affected portions of the 
longitudinal profiles could be removed. 

Cuts (1), (2), and (4) in Table [3] ensure a reliable reconstruction of the shower energy and the 
depth of shower maximum. However, cut (2) on the shower profile is loose enough to accept events 
which are not well-described by a Gaisser-Hillas function. This includes bumpy events, or profiles 
with a strong asymmetry. The purpose of cut (3) on shower energy is to heavily limit the number 
of lidar scans: from about two per 17 day FD measurement period during austral summer up to 
two per night during winter. The cut on the zenith angle (5) rejects overly inclined showers, which 
are problematic because the scan path may be so long that a lidar would not finish the StS within 
the maximum veto time window of four minutes. Finally, cut (6) excludes air showers which occur 
at close distances to the FD telescopes, because these do not need corrections for atmospheric 
transmission. The effects of the cuts are shown as a function of energy in Fig. [T5|. 
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Figure 16. Candidate event distribution for Shoot-the-Shower estimated by using hybrid events collected 
between January 2006 and August 2011. The effect of the selection cuts is shown: 99.92% of the original 
events are discarded. 
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Table 4. A list of quality cuts for the StS "double-bump" trigger. The fraction of rejected events were 
calculated using hybrid events observed between 2004 and 2010, after removal of nights contaminated by 
clouds and aerosols. During online running, the fractions of rejected events are substantially larger because 
clouds and aerosols cannot currently be removed in real time. 



Events which are analyzed by the lidar run control program are also removed if the corre- 
sponding shower has occurred more than 10 minutes in the past. This is to ensure that the StS 
measurements accurately describe the distribution of clouds and aerosols when the shower was ob- 
served. During times when both the online reconstruction and the lidar system were operational, 
no events had to be rejected because of this criterion. 

5.1.3 Anomalous Profile ("Double-Bump") StS Trigger 

In March 201 1, a second StS trigger was implemented to aid in the search for anomalous longitudi- 
nal profiles. This search is motivated by recent simulations which indicate that in a small fraction 
of air showers, leading particles can penetrate deeply into the atmosphere before interacting and 
creating another maximum in the shower profile [1T3TI . The longitudinal profile of such showers will 
have two peaks - hence "double -bump" showers - which can be fit using a sum of two Gaisser- 
Hillas functions. The fit yields two values of X max (X max i and X maX 2) for the two peaks in the 
profile. 
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The goal of StS in this analysis is to help discriminate double-peaked fluorescence profiles 
caused by hadronic interactions from the much larger background of double-bump showers caused 
by scattering due to aerosols and clouds. Based on hybrid events observed between January 2004 
and December 2010, a set of simple cuts has been developed to select double-bump events for StS. 
The cuts are listed in Table and simply require that: (1) a sum of two Gaisser-Hillas functions can 
be fit to the longitudinal profile; (2) both X max ,i and X maXj 2 are within the FD telescope field of view; 
and (3) fitting the profile with two Gaisser-Hillas functions results in a substantial improvement 
over a single Gaisser-Hillas fit. 

We note that the double-bump trigger is not energy-dependent; in principle low and high- 
energy showers can give rise to double-peaked profiles. Therefore, to prevent the energy cuts 
described in Section |5.1.2| from eliminating double-peaked events, the double-bump trigger has 
been set to supersede the standard StS trigger. In addition, we note that the cuts in Table || were 
tuned on a data set in which nights with heavy cloud and aerosol contamination were removed 
using the atmospheric databases. Therefore, the trigger is effective insofar as clouds and aerosols 
can be removed in real time. This issue will be discussed further in Section E2- 

5.2 Results 

Between January 2009 and October 2011, 112 air showers triggered an StS scan and were suc- 
cessfully reconstructed offline. Of these showers, 58 triggered telescopes at one FD site; 40 were 
observed in stereo mode at two FD sites; eight were observed at three FD sites; and six events were 
observed at all four FD sites. In total this sample comprises 186 individual fluorescence profiles. 
The reconstructed ground impact (or core) locations of the 112 air showers are shown in Fig. [T7J, 
superimposed on the SD array. The energies of the events, reconstructed offline with all available 
calibration and atmospheric databases, are shown in Fig. [TE[ 

Among the notable features in Figs. |T7| and |TS| are showers with core locations not contained 
inside the boundaries of the array, as well as several very high-energy events (> 10 eV). These 
features are due to events in which the SD "signals" corresponding to the FD longitudinal profiles 
were identified by the Off line software as coincident noise. Therefore, the showers were recon- 
structed in FD-monocular mode using no SD data, rather than hybrid mode. Such events are easily 
removed from the data by applying cut (4) listed in Table [I]. Applying this cut removes the showers 
with cores not contained in the SD array, as well as the high-energy events in Fig. |TE|. The cut was 
intentionally omitted from the StS trigger because FD-only events are still useful for systematic 
studies if they are observed in stereo mode (cf. [B4T1 ). 

5.2.1 Air Shower Analysis using StS data 

Our main interest in this study is the effect of StS on the standard hybrid analyses, and so we cut all 
events not reconstructed in hybrid mode. This reduces the sample to 89 events (146 FD profiles). 
We also require that the remaining FD profiles have a corresponding StS scan from the lidar station 
located at the same FD site. E.g., an air shower observed at Los Leones must have an StS scan 
from the lidar at Los Leones. This requirement reduces the event sample to a final size of 62 events 
(86 FD profiles). The reduced statistics are caused by down time of individual lidar stations during 
repair and maintenance periods. 
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Figure 17. Core positions of air showers at the array (cf. Fig. |7|) which triggered StS and could be 
reconstructed offline. The shot statistics are: 20 events at Los Leones (circles), 28 at Los Morados (squares), 
28 at Loma Amarilla (triangles), and 33 at Coihueco (inverted triangles). 
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Figure 18. Distribution of the energies of StS events, reconstructed offline. The light gray histogram shows 
events reconstructed as hybrid showers (removing "monocular" FD events with coincident noise hits in the 
SD) and with X max in the field of view of the FD telescopes. The dark gray histogram indicates the events 
which activated the double-bump trigger. 



The StS scans have been analyzed in order to find incidental clouds along the shower path. 
An automatic cloud detection algorithm used to estimate cloud coverage at each FD site ITJ5TI has 
been adapted for these scans by implementing a progressive re-binning of the signal trace. This 
allows for an extension of the lidar range up to 30-35 km, depending on atmospheric conditions. 
An example StS scan is shown in Fig. [HJ 
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Of the 62 StS scans, clouds have been detected in 27. Since the quality cuts used in most hybrid 
analyses require periods with less than 20% cloud coverage, the StS is particularly important if it 
identifies clouds during otherwise clear periods. It appears that such cases are rather uncommon; 
of the 27 StS scans affected by clouds, only two occurred during periods of low cloud coverage. In 
both cases, the clouds observed were at rather high altitudes (7 km and 10 km, respectively) and did 
not appear to affect the transmission of light between the shower axis and the observing telescopes. 




distance [m] 

Figure 19. Lidar scan from Coihueco after the StS trigger received at GPS second 960683560. Altitude is 
measured from the lidar station, which is located about 1 700m above sea level. The color scale shows the 
logarithm of the backscattered laser light after correction for the r~ 2 decrease in the return signal. Bright 
areas correspond to reflections from clouds. For more details about lidar signal processing, see fflQl . 

5.2.2 Analysis of Double-Bump Triggers 

The StS double-bump trigger began operating during the FD measurement period of February- 
March 2011. This was a period marked by significant broken cloud cover nearly every night of 
the data taking. As discussed in Section |5.1.3| and Table |], the double-bump cuts were tuned on 
a cloud-free data set, making the double-bump trigger susceptible to false positives in the real 
data where clouds were not removed. In fact twenty double-bump triggers were recorded and shot 
during this measurement period, an average of 1.3 per night. Inspection of the data confirmed that 
all of the showers were affected by atmospheric scattering. 

A representative double-bump event, observed at Loma Amarilla, is shown in Figure ^(J The 
fluorescence light longitudinal profile (Fig. ^) exhibits spikes at slant depths of 650 g cm~ 2 and 
750 g cm~ 2 . For this shower geometry, the spikes correspond to altitudes of about 3.6 km and 
4.7 km above the FD site Loma Amarilla, which is at an altitude of about 1.48 km above sea level. 
Such spikes are characteristic of multiple-scattering of light as the air shower enters a cloud layer. 
Inspection of the StS profile recorded by the Loma Amarilla lidar station confirms the presence of 
a reflective cloud layer in the shower-detector plane at the expected altitude that is responsible for 
the spikes. 
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Figure 20. Left: Profile of energy deposit measured at Loma Amarilla, 5 March 2011, showing the shower 
track and dE/dX profile of a double-bump trigger. Right: Lidar scan at Loma Amarilla corresponding to this 
"double-bump" event. A layer of broken clouds is clearly visible in the light path between the FD and the 
shower track. 

Unfortunately, an analysis of the first double-bump profiles reveals that all of the triggers 
were clearly spurious events created by obscuring clouds. In order to avoid wasting valuable time 
shooting low-energy showers affected by clouds, it is clear that some kind of real-time identification 
of coverage will be necessary. This may already be possible with existing lidar data. The cloud- 
detection algorithm used on the AutoScan data and described in IB5H is both fast and robust, and 
could be applied online. An online cloud coverage estimate could then be used to suppress StS after 
double-bump triggers whenever the coverage was over some minimum threshold, such as 20%. The 
real-time identification of cloud coverage and its application to StS is currently under study. 

6. Rapid Monitoring with FRAM 

The (F/Ph)otometric Robotic Atmospheric Monitor, or FRAM telescope, is capable of carrying 
out rapid monitoring observations. FRAM is operated as a passive scanner - i. e., observations do 
not require the use of lasers or light flashers - and so it does not introduce any dead time into the 
FD data acquisition. Therefore, unlike the lidars, there are no limitations on the use of FRAM in 
the rapid monitoring program. FRAM can be programmed to scan the shower-detector plane of 
interesting showers and record CCD images along the plane. Since the FRAM telescope is located 
close to the FD building at Los Leones, its use in the rapid monitoring program is limited only to 
hybrid air showers observed from Los Leones. From this location, sequences of CCD images are 
produced along the shower-detector planes of observed cosmic ray events. 

The FRAM telescope is equipped with two CCD cameras. A wide-field 4 (WF) camera is used 
to measure the atmospheric extinction along the shower-detector plane, and a narrow-field 5 (NF) 
camera is used to calibrate the WF images. The field of view of the WF camera is 240' (4°) in 
azimuth (aligned with right ascension) and 160' (2.67°) in elevation (aligned with declination). 
Hence, a shower traversing the whole field of view of an FD telescope is typically covered by 10 to 
20 CCD images (Fig. ^A§. The first image of the sequence is oriented along the axis of the cosmic 

4 Finger Lake Instrumentation (FLI) MaxCam CM8 with Carl Zeiss Sonnar 200 mm //2.8 telephoto lens. 
5 Since June 2010, we have used the Moravian Instruments CCD camera G2. Before June 2010, we used another FLI 
MaxCam CM8. Both cameras have been situated at the focus of the main 30 cm-diameter telescope. 
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Figure 21. Example of a shower track sampled by individual exposures of the CCD camera. The red 
rectangles show the approximate fields of view of the wide field camera images. The distortions of the 
shapes of the fields of view are due to the projection onto rectangular coordinates. The isolated rectangle in 
the upper left corner is the first image in the sequence, taken along the arrival direction of the cosmic ray 
shower. 



ray air shower to search for optical transients that could be associated with event, assuming the 
primary cosmic ray was a neutral particle. The field of view of the NF camera is about 25' (0.4°) 
in right ascension and 11' (0.3°) in declination, and is centered within the WF camera image. 

The images recorded with FRAM are analyzed automatically. First, stars in the image are 
identified and their observed magnitudes are compared with values from a catalog, allowing com- 
putation of the measured atmospheric extinction along the line of sight to each star. As there are 
typically several hundred identifiable stars in each image, it is possible to monitor the extinction 
along the shower track with high angular resolution. Variations of the resulting extinction coeffi- 
cient reveal the presence of clouds or prominent aerosol layers. 

Currently, the rapid monitoring observations of FRAM are carried out using the Johnson- 
Cousins B filter (central wavelength 435 nm; FWHM 100 nm) of the UBVRI set of astronomical 
filters [BHl . Both cameras are equipped with the same set of filters. For the comparison with FD 
observations, the Johnson-Cousins U filter (central wavelength 360 nm; FWHM 64 nm) would 
be more suitable, but the transmission of the WF camera of the FRAM telescope decreases very 
quickly below 400 nm, and in the U filter we typically detect only several stars, too few for a 
successful computation of the extinction along the shower-detector plane. 

Following astronomical convention, the light extinction is obtained by calculating the differ- 
ence between the measured magnitude m \, s of observed stars and the tabulated magnitude m ta b 
available in star catalogs [B7H . This difference depends on the optical path length through the atmo- 
sphere, known in astronomy as the airmass AM. The airmass is a zenith-dependent quantity similar 
to slant depth that is expressed with respect to the total atmospheric overburden observed along a 
vertical path between sea level and the top of the atmosphere. From these quantities, the extinction 
can be expressed as an extinction coefficient k = (m t, s — m ta b) /AM. The extinction coefficient can 



-30- 



then be transformed to optical depth using the expression 6 

t = \/T00 k = Q924k (61) 
e 

The relatively straightforward analysis of the CCD images is an advantage of the FRAM rapid 
monitoring program, because the analysis can be almost completely automated. Currently, the 
analysis is carried out offline by the telescope operator, but a fully automated analysis of the FRAM 
rapid monitoring data is foreseen in the near future. 



6.1 FRAM cuts for rapid monitoring 

The passive nature of the FRAM observations allows for few constraints in the selection of suitable 
events relative to the cuts applied for StS (see Sec. |5TTD . Currently, three sets of cuts are applied 
to focus on different types of analyses. The cut parameters can be modified interactively using the 
RTS2 control software of the telescope [|3"%ll . even on very short time scales such as one night. 

The three sets of cuts are shown in Table 0. Set 1 has been chosen to match the standard selec- 
tion of high-quality hybrid events used in the analysis of the energy spectrum \UM . Set 2 applies the 
selection criteria used to identify very deeply-penetrating photon-induced showers, closely match- 
ing the cuts used to estimate the photon upper limit with hybrid events fl3"9H . Finally, the third set of 
cuts is a relaxed version of the standard quality selection. It is designed to exploit the full capability 
of the FRAM passive monitor, i. e., to perform rapid follow-up monitoring of a very large set of 
events without interrupting the FD DAQ. The only criterion that is more restrictive in set 3 than in 
set 1 is the number of triggered PMTs of the FD camera. Without this, set 3 would generate too 
many triggers. The higher number of PMTs ensures that showers are selected with long tracks that 
might be better covered by the series of CCD images. The three types of triggers are combined in 
a logical OR so that a shower fulfilling any one set will trigger rapid monitoring with FRAM. 

In March 2011, the FRAM trigger was updated to include a search for anomalous "double- 
bump" showers. As with lidar StS, the double-bump trigger includes an additional cut on the 
improvement in the fit of the shower profile when using two Gaisser-Hillas functions. To ac- 
commodate such observations, we lowered the energy threshold for the FRAM rapid monitoring 
triggers to 3 x 10 17 eV in March 201 1. However, a dedicated trigger filter for anomalous events is 
currently being implemented. We describe the search for anomalous shower profiles in Section |53| . 



6.2 Performance of the FRAM telescope 

The FRAM rapid monitoring program has operated since November 2009. For the first two months, 
it was running in a test mode. The cuts given in Table |5] have been applied since January 2010. 

To successfully analyze a CCD image, it is necessary to obtain its astrometry. This means 
identifying the stars recorded in the image using a catalog in order to obtain their exact positions 
and catalog magnitudes. The shower track is scanned by both CCD cameras at the telescope, but 
only the wide-field WF camera fully covers the shower track without gaps. Therefore, the data from 

6 Since the magnitude system is one of relative brightness, a star that has a magnitude five times greater than that of 
another star will have a light intensity 100 times greater. 
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1 

(standard) 


Set of Tuts 

2 

(photon showers) 


3 

(relaxed standard) 


1. 


Reconstructed £" 


> 10 18 eV 


> 10 18 eV 


> 10 18 eV 


2. 


Relative uncertainty Ge/E 


<0.2 


— 


<0.4 


3. 


Uncertainty of X max 


< 40 g cm" 2 


— 


< 80 g cm" 2 


4. 


Triggered PMTs 


— 


> 6 


> 10 


5. 


Profile quality: Xgh/^ 


< 2.5 


< 6 


< 10 


6. 


Comparison of GH and linear fit: 










/y 2 V 

Min AGH 


>4 


— 


> 1 




#Gh/ #Hn 




<0.9 


— 


n 
1 . 


Zenith angle 6 of air shower 


~ nr\o 

< /u 






8. 


FD telescope-shower viewing angle 




> 15° 




9. 


Hottest station-core distance 


< 2000 m 


< 1500 m 


< 2000 m 


10. 


Time elapsed since shower arrival 


< 600 s 







Table 5. Three sets of cuts for the FRAM rapid monitoring program. Since March 2011, the energy 
threshold for all three sets has been lowered to E > 3 x 10 17 eV. 



the narrow-field NF camera are only used to cross-check values from the WF camera. For analysis 
of the shower-detector plane, we require at least 5 images from the WF camera with astrometry. 

Between January 2010 and the end of July 201 1, the FRAM telescope received 1 038 triggers 
for rapid monitoring fulfilling at least one of the three set of cuts described in Table |5[ Of those, 586 
were successfully observed - i. e., at least 5 WF camera images with astrometry were obtained - 
with 242 of showers observed in 2010 and 344 in 201 1 7 . The much higher event rate in 2011 was 
achieved after an upgrade of the telescope control software in March 2011. To allow for more 
frequent observations, the energy threshold for the triggers in Table | was lowered to 3 x 10 17 eV. 

The overwhelming majority of the showers, 574 out of 586, passed the relaxed trigger condi- 
tion (set 3). The standard trigger (set 1) was more restrictive, passing 319 events, as was the photon 
trigger (set 2), which passed 305 events. The statistics of each trigger filter and the overlap between 
them are shown in Fig. ^2|. The overlap between the filters is relatively large; 236 showers passed 
all three trigger conditions. Only 1 1 events were exclusively passed by the standard cuts, and just 
one event was passed by the photon cut alone. However, the photon trigger was affected by a bug 
before July 201 1 that caused events to be lost, so we note that the passing statistics from set 2 are 
artificially low with respect to the other two filters. 

6.3 Results 

The output of the FRAM rapid monitoring analysis is a set of extinction coefficients {k} computed 
in the Johnson-Cousins B filter for each star detected in the WF camera image. A complete shower 

7 The observations of showers in periods of bad weather with many clouds do not have a sufficient number of images 
with astrometry. The rate of successful observations was lower in 2010 due to telescope software and hardware issues, 
and it has increased in 201 1. 
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Set 1 




Set 2 



Set 3 



Figure 22. Distribution of successful rapid monitoring observations by the FRAM telescope according to 
different sets of cuts as given in TableQ 

track is covered by 10-20 WF CCD images (Fig. ^3]), and so we obtain several hundred to several 
thousand stars with computed extinction coefficients per track (Fig. 0). 

To find obscurations along the line of sight to the shower, we compare the extinction data to a 
"reference" profile of the extinction in a clear sky. The clear-sky profile is estimated by assuming 
a uniform atmosphere in which the extinction depends only on the airmass. This gives a zenith- 
dependent reference curve which can be compared to the extinction coefficients determined using 
the WF images. For example, the extinction coefficients in Fig. show a dramatic deviation from 
the clear-sky reference between 10° and 15° elevation due to the presence of the clouds visible in 



6.3.1 Air Shower Analysis using FRAM rapid monitoring data 

To determine whether or not the FRAM observations can be used to identify prominent changes in 
the extinction which affect the air shower reconstruction, we have searched the 242 events mon- 
itored with FRAM in 2010. Within this sample, 13 extinction profiles indicated the presence of 
clouds. 

After inspection of the corresponding longitudinal profiles recorded with the FD telescopes, 
we have found that 7 of the 13 events also passed the standard quality filter, which includes a tight 
cut on the profile reconstruction (^Q H /ndof < 2.5) that tends to remove events affected by clouds. 
Hence, out of 242 showers which satisfy all standard criteria for high-quality hybrid observations, 
7 events (3%) were clearly recorded with clouds in the shower-detector plane. 

We note at this point that there are further cuts applied on data for physics analyses, e.g., mass 
composition studies or building an energy spectrum, which remove events affected by clouds. The 
most important requires the cloud coverage in the sky above each telescope to be below 20% during 
a given measurement hour. The cloud coverage is estimated from the lidar AutoScan, in which the 
lidar stations continuously sweep the high- zenith regions above the FD telescopes. Using the 20% 



Fig. 



-33- 



i : • • 


4 


7 


10 








'.0$.. 


• ■' . ' • 






D) 50 
CD 

2, 


7 ■ 1 






••■ . >• 


■ '.' ■ <" 




g 

5 40 
_CD 
CD 


- 




2 • / 


5 


8 . ; 

■ ■ * 




30 


- 


■ 


. • • ' ' '" ' 












■ 






. ' * 




20 


- 


V 


3 % 


6 


9'- ' '. 

■ 




10 




I I 1 T I T 1 


■ 

_ 8 
■ 

■ 

_10 
■ 

■ 

t T I ! 1 ! ! I ! ! T 1 I T T T 1 










110 120 


130 140 150 160 














azimuth [deg] 



Figure 23. Left: A series of eleven wide-field (WF) CCD images recorded during the sampling of a shower 
track. Right: the alt-azimuth coordinates of the centers of each of the eleven images shown in the left panel. 
Note that the images have not been Hat- fielded, causing the image borders to appear darker than the centers. 
Also note that the WF camera is tilted about 20° from vertical (see image 11, where the Andes Mountains 
fill the bottom half of the image). In images 5-7, an optically thick cloud is observed to block the stars. 
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Figure 24. The analysis of the series of images shown in Fig. [2^. The computed extinctions for the 
individual stars are shown along the extinction fit for the clear sky. The prominent drop between 10° and 
17° clearly indicates the presence of the optically thick cloud. 
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cloud coverage cut, we find that 10 of the 13 cloud-affected FRAM observations will be removed 
from physics analyses. Of the remaining three events, two were observed during periods of broken 
cloud coverage with non-zero coverage in the neighboring measurement hours. However, one event 
was observed during a night in which the lidar data indicates cloud-free conditions during the whole 
night. After applying the cut on the profile reconstruction quality, only one of these three events 
passed the standard criterion. 

It may appear surprising that the lidar analysis could miss a prominent cloud layer clearly 
observed with the FRAM telescope, but we must note one critical limitation of the FRAM analy- 
sis. Because the extinction is determined using stellar magnitudes, the FRAM measurement only 
describes the overall atmospheric influence along the line of sight between the FD telescopes and 
the top of the atmosphere. Using the FRAM alone, it is not possible to determine whether or not a 
shower profile was observed behind a cloud or in the foreground. In order to determine whether or 
not a cloud seen by FRAM actually affected a fluorescence profile, it is necessary to look into the 
FD data themselves to see if the profiles are affected by spurious gaps (absorption) or spikes (mul- 
tiple scattering). Of the 13 cloud-affected events discussed above, inspection of the data suggests 
that 1 1 showers originated behind or partially inside the observed clouds. 

This study illustrates the trade-off between passive and active atmospheric monitoring for 
the FD calibration; the FRAM can operate during FD DAQ without constraints, but the resulting 
measurements do not have the very useful slant depth resolution of the lidar data. 

Nevertheless, the rapid monitoring observations performed with FRAM confirm that the stan- 
dard selection of hybrid events for physics analysis at the Pierre Auger Observatory is efficiently 
removing profiles affected by atmospheric scattering. The contamination of the high-quality stan- 
dard data set by the presence of clouds was found to be of order 1%. The problematic cases within 
this subsample will be further studied in order to improve future data processing. 

6.3.2 Analysis of the double-bump events observed by the FRAM telescope 

Because of the short operation of the double-bump rapid monitoring observations, only a few can- 
didates were analyzed for this purpose, and all of them are compatible with profiles containing 
clouds. As with StS, the anomalous profile trigger will need to be tuned using cloudy and clear data 
before it can become truly effective. 

7. Conclusion 

A rapid atmospheric monitoring program has been implemented and operated successfully at the 
Pierre Auger Observatory since early 2009. The online reconstruction has produced reliable data 
which have been used to trigger dedicated observations of the atmosphere. Among the showers of 
particular interest are those initiated by very high-energy cosmic rays, or those with unusual lon- 
gitudinal profiles. Since the start of rapid monitoring, there have been 52 dedicated balloon flights 
(until the termination of the balloon program end of December 2010), 112 lidar scans (through 
October 2011), and 586 observations with an optical telescope (through July 2011). The program 
has demonstrated that it is possible to perform targeted atmospheric monitoring based on real-time 
cosmic ray measurements. 



-35- 



The Balloon-the-Shower program was a useful tool for studying short-term variations in the 
altitude-dependent profiles of the main atmospheric state variables. Using actual atmospheric pro- 
files instead of monthly models in dedicated air shower analyses improves the accuracy of the 
Off line reconstruction. Unfortunately, this part of the rapid monitoring program imposed a large 
burden on the collaboration, and due to technical problems the observations fell short of the ex- 
pected rate. For these reasons, the program was discontinued at the end of 2010. However, the 
measurements have had a positive effect, as we have used BtS measurements to confirm the va- 
lidity of atmospheric profiles from the Global Data Assimilation System (GDAS). Profiles from 
GDAS, which have more regularity and better time resolution than the balloon flights, are now part 
of the standard Off line reconstruction. 

For the Shoot-the-Shower program, we have configured the lidar telescopes to search for show- 
ers affected by clouds during periods that might otherwise be flagged as cloud-free by the standard 
hourly lidar scans. A similar program has been implemented using the FRAM optical telescope, 
but without the operational limitations imposed on the lidar. Analysis of the StS and FRAM mea- 
surements indicate that it is relatively difficult to find events affected by atmospheric anomalies 
during periods that are identified as clear by the standard monitoring program. In other words, the 
rapid monitoring confirms that existing cuts on the quality of the shower profile reconstruction and 
on atmospheric conditions are effective at removing showers distorted by clouds and aerosol layers. 

Since the StS and FRAM measurements require much less human intervention than BtS, we 
expect that these programs will continue for at least several more years. Both programs have 
recently incorporated a trigger based on the search for anomalous double-bump shower profiles. 
Analyses of the early data indicate that some real-time restrictions of double-bump monitoring - 
especially during periods of high cloud coverage - are necessary to make this program effective. 
Real-time cloud monitoring should be possible using the standard lidar scans. A trigger filter based 
on total cloud coverage is currently under development. 

Finally, we have found that the rapid monitoring program can be extended easily to incorporate 
other instruments. It is particularly useful for measurements which must be restricted to avoid 
interference with the fluorescence telescopes. In 2012, a Raman lidar will be installed at the center 
of the Pierre Auger Observatory, and we expect that this instrument will become part of the rapid 
monitoring program. 
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